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Description 



MULTIPURPOSE VISION SENSOR SYSTEM 

Background of Invention 

[0001] jhe present invention relates to vehicle sensing systems. 

More particularly, the present invention relates to a 
method and system for performing multiple sensing sys- 
tem operations utilizing a single vision sensor. 

[0002] Various sensing systems currently exist for performing 

collision warning and countermeasure system operations, 
such as detection, classification, tracking, and relative 
distance and velocity estimation of objects within a close 
proximity of a host vehicle. Sensing systems also exist for 
performing other sensing system operations, such as 
windshield wiper and defogger operations, occupant 
sensing and tracking operations, and adaptive cruise con- 
trol operations. 

[0003] Collision warning and countermeasure systems operations 

include providing a vehicle operator knowledge and 
awareness of vehicles and objects that are within a close 
proximity of the host vehicle to prevent colliding with 



those objects. Countermeasure systems exist in various 
passive and active forms. Some countermeasure systems 
are used to aid in prevention of a collision, others are 
used to aid in the prevention of injury to a vehicle opera- 
tor. 

[0004] Certain collision warning and countermeasure systems are 
able to sense an object within close proximity of the host 
vehicle and warn the host vehicle operator, such that the 
operator can take precautionary steps to prevent a colli- 
sion or injury. Other collision warning and countermea- 
sure systems activate passive or active countermeasures 
such as airbags, load limiting seatbelts, or brake control 
whereby the system itself aids in preventing a collision or 
injury. 

[0005] Windshield wiper and defogger operations include clear- 
ing a windshield to provide a clear view of an environment 
forward of a vehicle. Moisture sensors exist for detecting 
mist, rain, or condensation on a vehicle windshield. In re- 
sponse to the amount of moisture sensed within a given 
area of the windshield, wiper speed may be adjusted and 
defoggers may be utilized to clear the windshield. 

[0006] Occupant related operations include detecting occupant 
characteristics determining which safety system counter- 



measures to perform and adjusting times and rates of tlie 
countermeasures. Example countermeasures tliat may be 
enabled are seat belt pretensioners and airbags. Occupant 
characteristics may include occupant positioning within a 
seat, occupant size, or other known occupant characteris- 
tics. 

[0007] Adaptive cruise control operations include adjusting a 

host vehicle traveling speed to maintain a safe operating 
distance between the host vehicle and a target vehicle. 
Other similar sensing systems also exist, such as lane de- 
parture and lane-keeping systems, which monitor lane 
markers or roadway lane designating lines and provide 
warnings when the host vehicle is not maintaining travel 
within a current operating lane. 

[0008] The various sensing systems may be ultrasonic, radar, li- 
dar, or vision-based. In order to perform each of the vari- 
ous sensing system tasks many of the stated sensors are 
distributed throughout the host vehicle. Each of the sen- 
sors increases vehicle manufacturing and maintenance 
costs. 

[0009] Thus, there exists a need for an improved sensing system 
that minimizes the number of sensors and related vehicle 
components, system complexity, and vehicle manufactur- 



ing and maintenance costs involved tlierein. 
Summary of Invention 

[0010] jhe present invention provides a metliod and system for 

performing multiple sensing system tasks or operations. A 
multipurpose sensing system for a vehicle is provided and 
includes an optic that is directed at multiple viewing ar- 
eas. A vision sensor is coupled to the optic and generates 
multiple object detection signals corresponding to the 
viewing areas. A controller is coupled to the vision sensor 
and generates multiple safety system signals in response 
to the object detection signals. 

[0011] One of several advantages of the present invention is that 

it provides a sensing system that is capable of monitoring 
multiple viewing areas using a single sensor. In so doing, 
the present invention minimizes the number of sensors 
needed to perform multiple sensing system operations. 
The present invention is capable of performing the sens- 
ing system operations that are traditionally performed by 
multiple separate sensing systems using a single sensor. 

[0012] Another advantage of the present invention is that it uti- 
lizes a single controller for operational control of the 
sensing system. The controller is capable of controlling 
multiple sensing system operations, thus further minimiz- 



ing number of components, costs, and weight of a sens- 
ing system. 

[0013] jhe present invention itself, togetlier witli attendant ad- 
vantages, will be best understood by reference to the fol- 
lowing detailed description, taken in conjunction with the 

accompanying figures. 
Brief Description of Drawings 

[0014] For a more complete understanding of this invention ref- 
erence should now be made to embodiments illustrated in 
greater detail in the accompanying figures and described 
below by way of examples of the invention wherein: 

[0015] Figure 1 is a block diagrammatic view of a multipurpose 
sensing system for a vehicle in accordance with an em- 
bodiment of the present invention. 

[0016] Figure 2 is a cross-sectional component and viewing area 
diagram of a bi-focal optic and a vision sensor in accor- 
dance with an embodiment of the present invention. 

[0017] Figure 3 is a cross-sectional component and viewing area 
diagram of the bi-focal optic, the vision sensor, and a re- 
flective device in accordance with another embodiment of 
the present invention; and. 

[0018] Figure 4 is a logic flow diagram illustrating a method of 

performing multiple sensing system operations within a 



vehicle in accordance witli an embodiment of tlie present 

invention. 
Detailed Description 



[0019] In tlie following figures the same reference numerals will 

be used to refer to the same components. The present in- 
vention may be adapted and applied to various sensing 
systems including: collision warning systems, collision 
avoidance systems, parking-aid systems, reversing-aid 
systems, passive countermeasure systems, adaptive cruise 
control systems, lane departure systems, lane-keeping 
systems, windshield clearing systems, or other systems 
known in the art. 

[0020] In the following description, various operating parameters 
and components are described for multiple constructed 
embodiments. These specific parameters and components 
are included as examples and are not meant to be limit- 
ing. 

[0021] Additionally, in the following description, the term "per- 
forming" may include activating, deploying, initiating, 
powering, and other terms known in the art that may de- 
scribe the manner in which a countermeasure or a comfort 
and convenience feature may be operated. 

[0022] As well, in the following description, various countermea- 



sures are discussed. The countermeasures may be re- 
versible or irreversible. Reversible countermeasures refer 
to countermeasures that may be reset to their original 
form or used repeatedly without a significant amount of 
functional deficiency, which may be determined by a sys- 
tem designer. Irreversible countermeasures refer to coun- 
termeasures such as airbags that, once deployed, are not 
reusable. 

[0023] Furthermore, a countermeasure signal may include infor- 
mation pertaining to the above-stated reversible and irre- 
versible countermeasures or may include other informa- 
tion, such as collision warning information, and parking- 
aid or reversing-aid countermeasure information. For ex- 
ample, the countermeasure signal may contain object de- 
tection information, which may be used to indicate to a 
vehicle operator the presence or close proximity of a de- 
tected object. 

[0024] In addition, the term "object" may refer to any animate or 

inanimate object. An object may be a vehicle, a pedes- 
trian, a lane marker, a road sign, a roadway lane designat- 
ing line, a vehicle occupant, window moisture, or other 
object known in the art. 

[0025] Also, although the present invention is primarily described 



with respect to a single sensor, in applications where a 
single sensor is infeasible or undesirable additional sen- 
sors may be utilized and may be utilized in a similar fash- 
ion. 

[0026] Referring now to Figure 1, a block diagrammatic view of a 

multipurpose sensing system 10 for a vehicle 12 in accor- 
dance with an embodiment of the present invention is 
shown. The vision sensing system 10 includes a multi-fo- 
cal optic 14 coupled to a vision sensor 16. The vision sen- 
sor 16 in conjunction with the lens 14 is capable of moni- 
toring multiple viewing areas 18, as are best seen in Fig- 
ures 2 and 3. A signal processor 20 is coupled to the vi- 
sion sensor 16 and formats object detections signals gen- 
erated by the vision sensor 16 for further processing by a 
controller 22. The controller 22 in response to the object 
detection signals performs various sensing system opera- 
tions. 

[0027] A safety system 24 and an indicator 26 are coupled to the 
controller 22. The controller 22 may enable various pas- 
sive countermeasures 28 and active countermeasures 30 
within the safety system 24 in response to the object de- 
tection signals. The controller 22 as part of a counter- 
measure may generate a warning signal and communicate 



the warning signal to veliicle occupants via tlie indicator 
26. 

[0028] The vision sensor 16 may be a camera, a cliarged-coupled 
device, an infrared detector, a series of pliotodiodes, or 
otiier vision sensor l<nown in tlie art. Tlie vision sensor 16 
may be located in various locations on the vehicle 12. 

[0029] jhe signal processor 20 receives and converts the object 
detection signals, generated by the vision sensor 16, into 
a proper format for reception by the controller 22. The 
signal processor 20 may include analog-to-digital con- 
verters, filters, or amplifiers, as well as other signal condi- 
tioning components known in the art. The signal proces- 
sor 20 may be part of the vision sensor 16 or the vehicle 
controller 22, may be a stand-alone device, as shown, or 
may be some combination thereof. 

[0030] Jhe controller 22 may be microprocessor based such as a 
computer having a central processing unit, memory (RAM 
and/or ROM), and associated input and output buses. The 
controller 22 may be an application-specific integrated 
circuit or be formed of other logic devices known in the 
art. The controller 22 may be a portion of a central vehicle 
main control unit, an interactive vehicle dynamics module, 
a restraints control module, a main safety controller, or 



may be a stand-alone controller as shown. 
[0031] jhe controller 22 may perform various different sensing 
system operations including adaptive cruise control, lane- 
keeping control, lane-departure control, window clearing 
control, collision avoidance control, countermeasure con- 
trol, or other sensing system operations known in the art. 
The operations may be performed sequentially or simulta- 
neously. 

[0032] Adaptive cruise control is used for monitoring objects for- 
ward of the vehicle 12 and for maintaining a safe prede- 
termined distance away from the detected objects to pre- 
vent collision therewith. When adaptive cruise control is 
active the controller 22 may warn the vehicle operator of 
an impending object or perform a countermeasure as to 
alter the speed of travel of the vehicle 12. 

[0033] Lane-keeping and lane-departure control refer to when 
the controller 22 monitors lane markings or roadway lane 
designating lines and warns the vehicle operator when the 
vehicle 12 is exiting a current lane of travel or is directed 
to exit the current lane of travel. The controller 22 may 
perform a countermeasure to maintain the current lane of 
travel, such as controlling the vehicle steering to adjust 
direction of travel of the vehicle 12. 



[0034] Window clearing control refers to performing tasks to 

clear a window, thus, providing a clear view of an environ- 
ment forward of the vehicle 12. The vision sensor 16 in 
conjunction with the multi-focal optic 14 detects and 
senses the amount of mist, rain, or condensation on a 
window of the vehicle 12, such as the windshield 32. In 
response to the amount of moisture sensed within a given 
area of the window, the controller 22 may adjust wiper 
speed of a wiper system 34 or activate or increase flow 
rate of defoggers within a defogger system 36. 

[0035] Countermeasure control may include occupant related op- 
erations, such as detecting occupant characteristics, de- 
termining which of the countermeasures 28 and 30 to 
perform, and adjusting times and activating rates of the 
countermeasures 28 and 30. The occupant characteristics 
may include occupant positioning within a seat, occupant 
size, or other known occupant characteristics. 

[0036] jhe controller 22 determines which of the sensing system 
operations to perform. The controller 22 while performing 
one or more of the sensing system operations may deter- 
mine whether to perform one or more of the countermea- 
sures 28 and 30 and indicate to the vehicle operator vari- 
ous object and vehicle status information. Depending 



upon relative positions, velocities, and accelerations of the 
detected objects, the controller 22 may also determine 
whether to indicate to the vehicle operator of a potential 
collision or may perform a countermeasure, as needed, so 
as to prevent a collision, mitigate a potential injury, or 
prevent the vehicle 12 from traveling outside a current 
lane of travel. 

[0037] jhe safety system 24 may be coupled to the controller 22 
and include object detection sensors, such as the vision 
sensor 16, and the signal processor 20. The safety system 
24 may include the countermeasures 28 and 30 as well as 
other safety system components, systems, or controllers 
known in the art. 

[0038] Jhe passive countermeasures 28 may include internal air 

bag control, seatbelt control, knee bolster control, head 
restraint control, load limiting pedal control, load limiting 
steering control, pretensioner control, external air bag 
control, pedestrian protection control, and other passive 
countermeasures known in the art. 

[0039] Jhe active countermeasures 30 may include brake control, 
throttle control, steering control, suspension control, 
transmission control, and other vehicle control systems. 
The controller 22 may signal the vehicle operator via the 



indicator 26 of an impending potential collision so that 
the vehicle operator may actively perform a precautionary 
action, such as applying the brakes or steering to prevent 
a collision. 

[0040] The indicator 26 is used to signal or indicate a safety sys- 
tem signal, which may include a warning signal, a colli- 
sion-warning signal, a countermeasure signal, or an ob- 
ject identification signal in response to the object detec- 
tion signals. The indicator 26 may include a video system, 
an audio system, an LED, a light, a global positioning sys- 
tem, a heads-up display, a headlight, a taillight, a display 
system, a telematic system, or other indicator known in 
the art. The indicator 26 may supply warning signals, col- 
lision-related information, lane departure and lane- 
keeping information, external-warning signals to objects 
or pedestrians located outside of the vehicle 12, or other 
pre and post collision information. 

[0041] jhe multipurpose system 10 may be located in various lo- 
cations in the vehicle 12. The multipurpose system 10 
may be located within an overhead console 38, near any 
vehicle window 32, near a vehicle bumper 40, internal to 
or external from a vehicle cabin 42, or in various other lo- 
cations known in the art. 



[0042] Referring now to Figures 2 and 3, cross-sectional compo- 
nent and viewing area diagrams of tlie optic 14 and tlie 
vision sensor 16 in accordance witli multiple embodi- 
ments of the present invention are shown. The optic 14 
may be directed at any number of viewing areas 18 and 
have any number of focal points 50. Viewing area bound- 
aries are represented by dashed lines 52. In the embodi- 
ments of Figures 2 and 3 the optic 14, is a bi-focal optic 
and has an upper portion 54 and a lower portion 56. The 
upper portion 54 has a first focal point 58 and corre- 
sponds to a first viewing area 60, with respect to the em- 
bodiment of Figure 2, and to a first viewing area 60", with 
respect to the embodiment of Figure 3. The lower portion 
56 has a second focal point 62 and corresponds to a sec- 
ond viewing area 64. The first viewing area 60 is utilized 
to monitor moisture levels on a vehicle window 32. The 
first viewing area 60" is utilized to monitor vehicle occu- 
pant characteristics. The second viewing area 64 is uti- 
lized to monitor objects external to and in a close prox- 
imity with the vehicle 12. 

[0043] In the embodiment of Figure 3, the first viewing area 60" 
corresponds with the viewing angle of a reflective device 
66. The lower portion 56 is directed at the reflective de- 



vice 66. The viewing area 60" lias a viewing boundary rep- 
resented by daslied lines 70, which may vary depending 
upon the application, type of reflective device, mounting 
locations and relative arrangements of sensing system 
components, and other viewing angle factors known in the 
art. In an embodiment of the present invention, the view- 
ing angle 68 is large enough to cover both driver and pas- 
senger seating areas of the vehicle 12. 

[0044] jhe reflective device 66 may be a wide-angle mirror, as 

shown, or other reflective device known in the art. The re- 
flective device 66 may be formed of various materials and 
may be of various sizes, styles, and shapes. 

[0045] jhe embodiments of Figure 2 and 3 are shown for illus- 
trative example only. Although the present invention is 
described with respect to a single multi-focal optic, and 
more specifically to a bi-focal optic, any number of optics 
or lenses may be utilized. The optics and lenses may be of 
various size, shape, and style and have any number of as- 
sociated focal points. As is known in the art, costs of op- 
tics and lenses are inexpensive relative to costs of vision 
sensors. 

[0046] Referring now to Figure 4, a logic flow diagram illustrating 
a method of performing multiple sensing system opera- 



tions within tlie veliicle 12 in accordance witli an embodi- 
ment of tlie present invention is sliown. 

[0047] In step 100, the vision sensor 16 in conjunction with the 
optic 14 and the reflective device 66, when used, monitors 
the viewing areas 18. As stated above, the viewing areas 
18 may be external, on, or internal to the vehicle 12. 

[0048] In step 102, the vision sensor 16 generates multiple ob- 
ject detection signals corresponding to the monitored 
viewing areas 18. Object detection signals may be contin- 
uously generated to inform the controller 22 of object 
status external to, on, and internal to the vehicle 12. 
When the reflective device 66 is utilized one or more ob- 
ject detection signals are generated corresponding to the 
viewing area 60". The object detection signals may be 
generated sequentially or simultaneously. 

[0049] In step 104, the signal processor 20 converts, modifies, 
and formats the object detection signal for reception by 
the controller 22. The signal processor may, for example, 
include an analog-to-digital converter to convert analog 
image signals into a digital format. 

[0050] In step 106, the controller 22 generates multiple safety 

system signals in response to the object detection signals. 
The safety system signals may include not only counter- 



measures related signals, as stated above, but may also 
include signals related to other sensing system controls, 
some of which are also stated above. 
[0051] In step 108, the controller 22 may determine occupant 

characteristics in response to the object detection signals. 
In determining occupant characteristics the controller 22 
may also determine whether a child safety seat is located 
within a vehicle seat of the viewing area 60". Step 108 
may be performed simultaneously with step 106, whereby, 
the controller 22 generates or modifies safety system sig- 
nals in response to the determined occupant characteris- 
tics. 

[0052] In step 110, the controller 22 may perform multiple sens- 
ing system operations in response to the safety system 
signals. The controller 22 may perform a countermeasure 
operation, a windshield clearing operation, an adaptive 
cruise control operation, or any of the other above- 
mentioned or known in the art sensing system operation 
in response to the safety system signals. For example, the 
vision sensor 16 may detect a moisture level on the wind- 
shield 32 to be above a predetermined level and in re- 
sponse thereto the controller 22 may activate or increase 
speed of the wiper system 34. As another example, the 



vision sensor 16 may detect an occupant tliat is small in 
size or a child safety seat within a seat system and decel- 
erate or prevent activation of an air bag near the occupant 
or safety seat. 

[0053] The above-described steps are meant to be illustrative 

examples; the steps may be performed sequentially, syn- 
chronously, simultaneously, or in a different order de- 
pending upon the application. 

[0054] jhe present invention provides a multipurpose sensing 
system that is capable of monitoring multiple viewing ar- 
eas using a single vision sensor and controller. The 
present invention in utilizing a single vision sensor and 
controller minimizes the number of sensors and sensing 
system components within a vehicle, sensing system 
complexity, and manufacturing and maintenance costs in- 
volved therein. 

[0055] While the invention has been described in connection with 
one or more embodiments, it is to be understood that the 
specific mechanisms and techniques which have been de- 
scribed are merely illustrative of the principles of the in- 
vention, numerous modifications may be made to the 
methods and apparatus described without departing from 
the spirit and scope of the invention as defined by the ap- 



pended claims. 



